3 research outputs found
Networked Federated Learning
We develop the theory and algorithmic toolbox for networked federated
learning in decentralized collections of local datasets with an intrinsic
network structure. This network structure arises from domain-specific notions
of similarity between local datasets. Different notions of similarity are
induced by spatio-temporal proximity, statistical dependencies or functional
relations. Our main conceptual contribution is to formulate networked federated
learning using a generalized total variation minimization. This formulation
unifies and considerably extends existing federated multi-task learning
methods. It is highly flexible and can be combined with a broad range of
parametric models including Lasso or deep neural networks. Our main algorithmic
contribution is a novel networked federated learning algorithm which is well
suited for distributed computing environments such as edge computing over
wireless networks. This algorithm is robust against inexact computations
arising from limited computational resources including processing time or
bandwidth. For local models resulting in convex problems, we derive precise
conditions on the local models and their network structure such that our
algorithm learns nearly optimal local models. Our analysis reveals an
interesting interplay between the convex geometry of local models and the
(cluster-) geometry of their network structure
Local Graph Clustering with Network Lasso
We study the statistical and computational properties of a network Lasso method for local graph clustering. The clusters delivered by nLasso can be characterized elegantly via network flows between cluster boundaries and seed nodes. While spectral clustering methods are guided by a minimization of the graph Laplacian quadratic form, nLasso minimizes the total variation of cluster indicator signals. As demonstrated theoretically and numerically, nLasso methods can handle very sparse clusters (chain-like) which are difficult for spectral clustering. We also verify that a primal-dual method for non-smooth optimization allows to approximate nLasso solutions with optimal worst-case convergence rate.Peer reviewe